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Abstract'. This paper evaluates the impact of Farmer Field Schools, an intensive participatory 
training program emphasizing integrated pest management. The evaluation focuses on whether 
participation in the program has improved yields and reduced pesticide use among graduates and 
their neighbors who may have gained knowledge from graduates through informal 
communications. The study utilizes panel data covering the period 1991-1999 in Indonesia. The 
analysis, employing a modified “difference-in-differences” model, indicates that the program did 
not have significant impacts on the performance of graduates and their neighbors. Several 
plausible explanations for this outcome are discussed, and recommendations for improvements 
are suggested. 
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BEST COPY AVAILABLE 



Sending Farmers Back to School: The Impact of 



Farmer Field Schools in Indonesia 

I. Background 

Agricultural extension and farmer education programs are key policy instruments for 
governments seeking to improve the productivity of agriculture while protecting the 
environment. Correspondingly, there is great interest in the impact of such public investments 
and in their financial viability. The experience of extension systems over the past few decades 
has been mixed. Some studies estimate high rates of return to the investment in extension 
(Birkhaeuser, Evenson, and Feder), or to farmer education (Jamison and Lau; Lockheed, 
Jamison, and Lau). Yet, many observers document poor performance in the operation of 
extension and informal education systems, due to bureaucratic inefficiency, deficient program 
design, and some generic weaknesses inherent in publicly-operated, staff-intensive, information 
delivery systems (Feder, Willett, and Zijp). One deficiency highlighted by researchers and 
practitioners is the tendency of many public officers dealing with the transmission of knowledge 
to conduct their assignment in a “top-down” manner. Often, the information that is conveyed is 
presented as a technological package comprised of recommended practices. This is perceived as 
a less effective method for improving knowledge as compared to more participatory approaches 
(Axxin; Braun, Thiele, and Fernandez). 

In recent years, a number of development agencies, including the World Bank, have 
promoted farmer field schools (FFS) as a more effective approach to extend science-based 
knowledge and practices to farmers. Though pioneered and first promoted by the Food and 
Agriculture Orgamzation (FAO) as a practical way of diffiising knowledge-intensive integrated 
pest management (IPM) concepts and practices for East Asian rice-based systems (Kenmore 
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1991; van de Fliert), the FFS has since evolved to include a much broader coverage of other 
farm-relevant topics in its curriculum. The FFS training program utilizes participatory methods 
“to help farmers develop their analytical skills, critical thinking, and creativity, and help them 
learn to make better decisions” (Kenmore 1997). Such an approach, in which the trainer is more 
of a facilitator, rather than an instructor, reflects a paradigm shift in extension work (Roling and 
van de Fliert). 

As an extension approach, the FFS concept does not require that all farmers attend FFS 
training. Rather, only a select number of farmers within a village or local farmers’ group are 
trained in these informal schools, which entail weekly meetings in a season-long tr ainin g course. 
However, in order to disseminate new knowledge more rapidly within the community, selected 
farmers receive additional training to become farmer-trainers, and are expected to organize field- 
school replications within the community, with some support fi-om public sources. Furthermore, 
all FFS graduates are encouraged to share their knowledge and experiences with other farmers 
within their local village and community organizations. These farmer-to-farmer diffusion effects 
are expected to bring about cost-effective knowledge diffusion and financial sustainability, issues 
that have hampered many public extension systems in both developed and developing coimtries 
(Quizon, Feder, and Murgai; Hanson and Just). 

However, there are risks inherent in the FFS approach. Given the complexity of the 
information conveyed in the training program, the ability to convey effectively to other farmers 
complex decision making skills through informal communications may be limited and may 
curtail the diffusion process. Moreover, the requirement fi"om trainers to abandon earlier top- 
down approaches to which they may have been accustomed in favor of a facilitation mode is a 
challenge that poses risks to the effectiveness of the program. 
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Previous studies evaluating the impact of FFS at the farm-level report significant impacts 
of program participation on farm-level yields and profits, and a decline in pesticide use. For 
example, studies by Nanta in Thailand and Ekneligoda in Sri Lanka claim that pesticide 
applications decreased with more IPM knowledge and FFS training, while rice yields increased 
by as much as 25%. A similar study by Ramaswamy, Shafiquddin, and Latif for Bangladesh 
notes that FFS-schooled farmers had 8-13% higher rice yields than their non-FFS counterparts. 
Similarly high impacts on farm profits are also reported by studies conducted in Vietnam, Ghana, 
Cote d’Ivoire and Burkina Faso (cited in Kenmore 1997). Increases of profits of 40% in Sri 
Lanka, 30% in Thailand, and 10-25% in China are cited in FAO (2000, p. 18). 

However, most previous studies have not accounted for econometric problems that arise 
in estimating program impact when the placement of the program across villages and the 
selection of farmers for participation in the program are not done at random. These, and other 
econometric issues discussed later in the paper, are likely to bias estimates of program impact. 
To our knowledge, the actual farm-level impact of this diffusion concept in an area where FFS 
operated on a large scale over a reasonable period of time has not been studied rigorously. 

This paper evaluates the impact of the FFS effort on farm-level outcomes (yields and 
pesticide use), focusing on Indonesia as a case study.' We employ a modified version of a 
“difference-in-differences” approach to evaluate program impacts, utilizing a panel household 
survey that includes information on both field school graduates and other farmers. The empirical 
strategy allows us to separately identify the direct impact on farmers who participated in the 
program, as well as the secondary impacts through farmer-to-farmer diffusion from graduates to 
other community members. It also enables us to control for concurrent exogenous events and 
interventions that are likely to affect farm performance over time, and to deal with selection 
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biases that arise due to the potential endogeneity of participation by farmers and biases inherent 
in non-random placement of training programs across communities. 

The Indonesian experience holds lessons for development agencies and governments in 
developing countries, who are being encouraged to expand and promote the FFS approach on a 
wider scale. Presently, there are large national FFS programs in several countries, and pilot or 
smaller scale programs in many countries of Asia, Africa, and Latin America. Of the various 
countries, Indonesia has had amongst the longest experiences with the FFS approach and 
therefore, provides the opportunity to study long-term secondary diffusion impacts that may not 
yet be manifest in other countries where FFS is relatively recent. The Indonesian FFS program 
is typical of large scale applications of the FFS concept, and has served as a model for programs 
elsewhere (Kenmore 1991). Thus insights of the performance of the program in Indonesia are of 
relevance in other countries considering national programs. Farmer field schools are a significant 
public expenditure undertaking if implemented on a large scale, as the cost per trained farmer is 
significant (Fleischer, Waibel, and Walter-Echols; Quizon, Feder, and Murgai) and a rigorous 
investigation of their impact is important. Furthermore, the effectiveness of reliance on farmer- 
to-farmer diffusion of complex information is of general interest in contemplating extension 
approaches. 

Our empirical results do not indicate that farmer field schools in Indonesia have induced 
significant improvements in yields or reduction in pesticide use by graduates relative to other 
farmers. Not surprisingly then, secondary diffusion effects on those exposed to graduates are also 
not significant. While we are unable to pinpoint a specific reason for these results, we advance 
some plausible arguments that might explain the absence of significant program impacts. 
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The rest of the paper is organized as follows. Section II develops a conceptual framework 
that underlies the empirical work. Section III describes the FFS program in Indonesia and the 
data set utilized in the study. In Section IV, we formulate the empirical specification and testable 
hypotheses, and in Section V present the key findings of the empirical analysis. The last section 
highlights the conclusions. 

II. Conceptual Framework 

FFS training aims to affect farmers’ knowledge, which can be interpreted broadly to 
include the possession of analytical skills, critical thinking, and ability to make better decisions, 
as well as familiarity with agricultural practices and understanding of interactions within the 
agricultural ecosystem. Improved knowledge is, in turn, reflected in farmers’ cultivation 
procedures, input decisions, and crop yields. 

Because knowledge and farmer performance can improve over time even in the absence 
of FFS training (through communication with other farmers, contact with other sources of 
information etc.), we model performance as a growth process. As improved knowledge is 
reflected in cultivation decisions, farm performance indicators such as yields and input use 
(denoted by Y) may also change over time, at some rate, say A field school can be expected 
to improve this performance growth rate for farmers by increasing farmers’ knowledge. 

Improvements in performance are expected for two groups of farmers. First, FFS 
training directly affects outcomes of field school “graduates ” (i.e., those who have participated in 
the training course), as it improves their ability to acquire and process information (through 
experimentation, interpretation of experience, etc). Second, the training may also indirectly 
affect farmers who are “exposed ” to FFS graduates. These are farmers who live in villages that 
have field school graduates, and therefore may have gained knowledge indirectly through 
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farmer-to-farmer diffusion. “ Control ” farmers, who live in villages where no farmer has received 
training, are unaffected by the program, and therefore continue to grow at the pre-program 
growth rate of a. 

In order to capture these features, consider a simple model in which an FFS program is 
introduced in some villages in cropping season T*, while other villages are not exposed to the 
program. Define a period between two points in time Tg and F/, where To< T*<T,, so that {T*-To) 
represents the number of cropping seasons before exposure to the program, and (Ji-T*) is the 
time period after exposure. In the villages without FFS, by definition, the number of seasons 
after program exposure is zero, and therefore T,=T*. Modeling a farmer’s performance as an 
exponential growth process implies that outcome Y (e.g., yield or pesticide use) at time T, can be 
expressed as: 

(1) 5^1 = Jo • 

where a measures the pre-program growth rate in performance for all households, is the 
growth rate of Y for exposed households after program exposure, and /j. is the growth rate of Y 
for graduates after they have participated in the program. Do and Du are dummy variables for 
graduates and exposed farmers, with control farmers being the omitted category. Any fixed (time 
invariant) factors that affect outcomes at either the household or the village level are already 
embodied in Vq , and thus do not need to be explicitly denoted. The variables X and Z denote 

vectors of farmer and village characteristics that also affect performance, y and S are the 
corresponding vectors of parameters, A denotes the differencing operator between periods To and 
T, (e.g., AX=XrXo), and 6 denotes the exponential operator. 



Figure 1 illustrates the growth process embodied in equation (1) for the three groups of 
farmers. Control farmers maintain the original rate of performance growth a throughout the 
period (7/ - T^. This rate of growth is expected to improve for both graduates and exposed 
households, after program exposure at time T*. In addition, the improvement among graduates 
is expected to be greater than among exposed farmers since graduates are intensively trained in 
FFS concepts. It is thus hypothesized that// > P> a. The impact of FFS on performance of 

graduates and exposed farmers can be measured by {ji-a) and (j3-a), respectively. While in 
the case of yields the program is expected to induce a higher growth rate, the reverse holds in the 
case of pesticides where the desired outcome is a reduction in use."* 

With the conceptual framework formulated, we turn to a description of the Farmer Field 
School program and its implementation in Indonesia. The econometric specification and testable 
hypotheses based on this conceptual firamework are developed in Section IV. 

III. Institutional Setting and Data 
FFS in Indonesia 

The typical FFS educates farmer participants on agro-ecosystems analysis, or what can be more 
generally described as integrated pest and crop management (IPCM), as it includes practical 
aspects of “... plant health, water management, weather, weed density, disease surveillance, plus 
observation and collection of insect pests and beneficials” (Indonesia National IPM Program 
Secretariat). Studies suggest that the information contained in the training program could, if 
property applied, lead to improved farm performance: As demonstrated in Settle et al., pest 
damage in rice could be contained using an integrated approach relying on biological and 
physical control while reducing costly pesticide applications (see also the review by Way and 
Heong). Pingali, Moya, and Velasco concluded that yield increases were possible if improved 



“second generation” cultivation practices were followed, and emphasized that training programs 
are particularly important for promoting such practices. Extensive supervised trials in Vietnam 
confirmed that the technology has the potential to reduce pesticide costs and improve yields, 
(P incus). 

The FFS approach relies on participatory training methods to convey knowledge to 
participants so as to make them into “confident pest experts, self-teaching experimenters, and 
effective trainers of other farmers” (Wiebers). An archetypal FFS entails some 8-12 weeks of 
hands-on, farmer experimentation and non-formal training during a single crop-growing season. 
A facilitator (typically government employee, but, in some cases, NGO or specially-trained 
farmer) leads this village-level program, focusing initially on problem-solving approaches in pest 
management, but also conveying knowledge pertaining to overall good crop management 
procedures and practices. Through group interactions, FFS participants sharpen their decision- 
making abilities, and are empowered by learning leadership, communication, and management 
skills (van de Fliert). 

Farmer field schools were introduced in Indonesia in 1989 for disseminating IPM 
technology among rice growers. Villages and farmer participants were purposively selected for 
inclusion in the program. Village accessibility and the presence of active farmer groups served 
as criteria for FFS locations. The more affluent and better informed farmers in these villages 
were invited to participate in the field schools (van de Fliert, p.l57). In 1994, the National IPM 
Training Project took over these pilot FFS activities and launched a nationwide FFS effort (with 
fimding from the World Bank) for promoting integrated pest management and improved crop 
cultivation, mainly for rice, but also including some non-rice crops. 



During the expansion phase, villages in which field schools were introduced were still 
non-randomly selected by Ministry of Agriculture officials, based on certain criteria (e.g., rice- 
growing, presence of a farmer group, cooperation of village leaders, accessibility of the village’s 
field school to potential participants). Certain criteria were used to select the field school 
participants as well (e.g., rice farmer, ability to read and write, ability to participate in FFS 
activities and discussions). This selection was done with assistance from village-level officials 
and farmer group leaders. The program’s strategy was not to train all farmers in the community, 
but rather to rely on the spread of knowledge through farmer-to-farmer diffusion. This included 
the formal use of farmer-facilitators, or select farmers identified dmmg FFS who were invited to 
attend a “training-of-trainers” program towards their becoming FFS facilitators themselves. 
Fluthermore, informal diffusion through farmers’ communications among themselves was 
expected to improve the knowledge within the wider village community which did not 
participate in the field schools. 

Since their inception in Indonesia, farmer field schools have remained separate activities, 
implemented in addition to the regular agricultural extension activities undertaken nationwide by 
the Ministry of Agriculture. There are some private sector-led extension efforts, including FFS 
activities run by NGOs and training activities sponsored by input supply companies, but these 
are very few and of limited coverage. By 1999, more than 500,000 farmers in tens of thousands 
of villages had been trained in field schools. Over 20,000 farmers had attended “training-of- 
trainers”. 



The data come from a panel survey of Javanese households conducted by the Indonesian 
Center for Agro-Socioeconomic Research (CASER) in April/May 1991 and again in June 1999. 



Data 
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The baseline sample included randomly selected rice-growing villages that had already been 
covered by the program, as well as villages that were not yet covered by the program but were in 
areas where the program was planned to be implemented. All villages were visited in the repeat 
survey, but our analysis focuses only on those villages that had not yet been exposed to an FFS at 
the time of the baseline survey in 1991. Five of these villages had not yet been served with an 
FFS program even by the time of the 1999 survey, and the 52 households from these villages are 
thus a control group. Of the 268 households from villages where a field school had been 
implemented between 1991 and 1999, only 1 12 had actually participated in the tr ainin g (whether 
administered by officials or by farmer-trainers) while the remaining 156 households had not 
attended a program, but had been potentially exposed to some of its effects through informal 
communications with graduates of the program.^ Therefore, our data allow us to separately 
identify the effects of FFS on graduates and exposed farmers. 

Another source of variation in program participation comes from the fact that field 
schools were first introduced in the sample villages at different times between the two surveys. 
Of the 21 program villages in the sample, 13 were first exposed to field schools within three 
years after the baseline survey (i.e., by April/May 1993). 5 villages were exposed the following 
year, and the remaining 3 were exposed after 1995/96. Therefore, our data also allow us to 
explicitly account for the length of pre- and post-program exposure for graduates and exposed 
farmers. 

The 1991 survey collected information on households’ farm operations and 
characteristics for the 1990/91 wet season and on their household attributes, activities and assets. 
The 1999 resurvey covered the wet season 1998/99 and included most of the questions of the 
baseline effort. To the extent possible, interviewers avoided changing the language and format 
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of the original 1991 questions in order to preclude any bias that may arise when later comparing 
responses from both panels. The 1999 survey, however, collected additional data, such as when 
the household participated in FFS training, more information about the co mmuni ty, and some 
retrospective details, such as information on when the community was first exposed to FFS, 
details pertaining to the training attended by FFS participants and follow-up activities. 

Table 1 reports selected descriptive statistics for the three household types. Household 
size and composition is similar across the three categories. However, consistent with non-random 
farmer selection, there are significant differences in asset ownership across the groups, with 
graduates owning more land and farm assets (e.g., sprayers), and being more educated than 
exposed households. In addition, graduate households are significantly more likely to participate 
in community activities and work as village officers. 

IV. Empirical Specification and Testable Hypotheses 

Any empirical analysis assessing the impact of FFS must take account of the special 
aspects of FFS program implementation. A commonly used approach to measure impact would 
be to use ordinary least squares (OLS), regressing farm-level outcomes on a variable indicating a 
farmer’s participation in the program and on other variables that could affect farm outcomes. 
This type of regression is effectively a single difference that compares mean outcomes of 
program participants to mean outcomes of non-participants in order to measure program impact. 

However, in the Indonesian context (as in other places where FFS programs have been 
established), two problems arise with such single difference comparisons based on cross- 
sectional data. First, the establishment of FFS programs in communities is determined by several 
factors, some of which are not observed by researchers and may be correlated with farm-level 
outcomes. For example, if villages where yields were the highest were selected into the program. 



cross-sectional comparisons of participants and non-participants may show that field schools are 
associated with higher yields. This is not really the effect of the program but instead comes fi-om 
the rules that guide the placement of the program. Second, farmers who are selected to 
participate in FFS are likely to be different fi-om other farmers, in ways that are imobserved to 
the researcher. For example, if more motivated farmers were more likely to be selected, 
comparing graduates to other farmers in an OLS regression would over-estimate the program’s 
impact on farm-level outcomes. 

Because of these features - non-random program placement and participant selection - it 
is well recognized that single difference comparisons of outcome measmes (e.g., between 
participants and non-participants) can give severely biased estimates of impact. A popular 
approach for addressing this problem is the difference-in-differences or double difference (DD) 
estimator, obtained by comparing the change in performance before and after the program for a 
treatment group (i.e., graduates or exposed households) to the change in performance over the 
same time period for a control group that is unaffected by the program. Since the DD estimator 
relies on comparing changes in outcomes between participants and non-participants, it is not 
affected by selection biases that arise from time-invariant household or village imobservables 
(Glewwe and Jacoby).^ 

The empirical specification for measuring FFS program impacts implied by the 
conceptual fi-amework laid out in Section II is a straightforward modification of the DD 
estimator. Using the same notation as earlier, the equation to be estimated is obtained by taking 
the natural log of equation (1) and rearranging: 

(2) A(ln Y) = a{r - T, )-h (t, - T* )-h (t, - T* )+ -h ^ 




*■1 

1 



4 



13 



where A(ln7) = (ln7, -ln7o) and (T* = Ti) for control fanners. Equation (2) can be estimated 

by regressing the change in log of performance indicator 7 between 1990/91 and 1998/99 on a 
set of variables that measure the length of pre and post-program exposure and a set of other 
household, village, and district controls.^ 

To see how equation (2) is related to the standard DD estimator, consider a case where all 
villages are exposed to FFS programs in the same season (i.e., T* is the same across all graduates 
and exposed farmers in the sample). In such a case, the length of exposure variables are the 
same across all exposed and graduate farmers, and therefore equation (2) collapses to a 
regression of the change in ln(y) on the exposed and graduate dummies (scaled up by the 
constant (Ty-T*)), and the set of other household, village, and district controls. In such a 
regression, // and P measure the change in Y over time for graduates and exposed households 
relative to the change in Y for control households - these are the typical DD estimators of 
program impact. 

Equation (2) modifies the DD estimator in order to account for the fact that FFS 
programs were introduced at different times across the sample villages. The underlying logic is 
that at the community level, diffusion of knowledge among individuals takes place over time. 
The longer the time interval between the time when some community members undertook 
training and the 1999 survey, the greater the spillover effects on the performance of non- 
graduates of FFS within the community. Similarly, for field school graduates, the longer the time 
elapsed since their training, the greater the opportunities to experiment with the new concepts 
and modes of decision-making, and thus the improvement in their performance is likely to be 
larger.* For these reasons, the length of time elapsed since the FFS was implemented is 
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explicitly incorporated by multiplying both the graduate and exposed participation dummies with 
the length of post-program exposure. 

Estimation of equation (2) in essence compares the performance growth rates of 
graduates and exposed farmers in the post-program period to the growth rate experienced by 
control farmers. It can identify parameters or (the pre-program growth rate in Y for all farmers), 
^ (the growth rate of Y for exposed farmers after FFS was implemented in their community), and 
H (the growth rate in Y for graduates after they imdertook training). Given these parameter 
estimates, the impact of FFS on graduates and exposed farmers is simply and 

respectively. 

Note that because program impact is estimated in first differences, any time-invariant 
unobservable farmer or village characteristics that may have affected farmer selection or 
program placement are differenced out of the regression, and therefore do not bias estimates of 
the program impact parameters. 

Another feature of equation (2) is that program impact on graduates and exposed 
households is estimated by comparing changes in their performance relative to changes in the 
performance of control households. Therefore, any variation in performance due to factors that 
affect all farmers, such as systemic climate changes, concurrent policy and price changes, 
nationwide non-FFS extension efforts or other secular trends, is also eliminated. In addition, in 
order to control for differences in extension efforts across regions at the sub-national level, we 
include district dummies in the estimation. This implies a ‘within-district’ estimation (i.e., 
graduates and exposed households are effectively compared to control households within the 
same district), and therefore the program impact estimates are not confounded with differences in 
access to extension efforts or other public services across districts. 



In the empirical work that follows, we focus on two performance indicators: yields and 
pesticide use. As pointed out earlier, some observers have claimed that improvements in these 
indicators were in fact obtained by the FFS program in Indonesia, although those assertions were 
not backed by rigorous studies (Kenmore 1997, FAO 2000). Other inputs, such as labor and 
fertilizers, are not included as performance indicators, as it is not clear a priori in which direction 
their use should change as a result of the program. In the case of yields, both graduates and 
neighbors are expected to accelerate their rate of improvement in yields, although the 
improvement for neighbors is not expected to be as high as graduates, who receive intensive 
training. Pesticides are a special area of focus of the FFS, and IPM principles are highlighted in 
the curriculum. It is thus expected that pesticide use will be reduced after attending the program. 

In order to test these hypotheses, in the case of the yield regressions, we perform the one- 
sided tests of and (J3>0^) against the null hypothesis of no difference in pre- and post- 

program growth rates. In the case of the pesticide regressions, the signs of the tests are reversed, 
since the training aims to induce lower pesticide use. 

V. Empirical Results 

Table 2 reports the means of the yields and pesticide expenditures in the two survey 
years, for the three groups of farmers. Several observations are pertinent: First, yields have 
decreased between 1991 and 1999 for the whole sample (and for each group). This is compatible 
with an overall drop in Java yields of 5% over the same time period, as reflected in national 
statistics also presented in Table 2.® The second observation is that the sample, which was drawn 
from FFS program areas, has consistently higher yields compared to the Java average, apparently 
because the FFS program focuses on the relatively more productive, better irrigated rice 
producing areas. Third, none of the differences in changes over time between the sub-groups in 
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either of the performance indicators are statistically significant.'® And finally, note that 
expenditures on pesticides have significantly increased for the sample and within each group. 
This is consistent with aggregate production figures for the rice pesticide market, which show an 
increasing trend in production (in terms of both active ingredient weight and value) during the 
early to mid-nineties (Oudejans). 

We turn now to multivariate analyses. Table 3 reports estimates of the pre-program and 
post-program growth rates of yields and pesticides using equation (2)." The estimates of pre- 
program growth rates reflect the trends reported in the descriptive statistics: on average, during 
the sample period, yields were declining and pesticide use increasing before the introduction of 
farmer field schools. Parameter estimates of post-program growth rates in yields and pesticide 
use suggest that these trends continued even after the introduction of FFS, both for graduates and 
exposed farmers. The question of interest is whether there was a significant acceleration 
(deceleration) in growth rates of yields (pesticides) for graduates and exposed farmers, relative to 
control farmers, after the introduction of field schools. This is tested in hypothesis tests A and B. 
Both tests find no evidence of impact, as there does not appear to be a significant difference in 
the trend growth rates in the post-program period for either graduates or exposed farmers. Test C 
which examines differences between graduates and exposed farmers in the post-program period 
similarly does not find any significant differences. 

Table 4 checks the robustness of these results by estimating a less restrictive specification 
than equation (2) that allows for the possibility that graduates have a different pre-program 
growth rate in outcomes compared to exposed and control farmers. Differences in pre-program 
growth rates might be expected if those selected to participate in field school training were 
superior performers not only due to fixed traits (e.g., superior managerial ability), but also due to 



dynamic traits (e.g., faster learners). Therefore, in Table 4 separate pre-program growth rate 
parameters are estimated for graduates ) versus other farmers (a, ) . 

The results in Table 4 indicate clearly that the graduates are not randomly selected to 
participate in FFS training. Test A, which compares the pre-program growth rates in outcomes 
for graduates with other farmers indicates that graduates had significantly higher yield growth 
and lower pesticide use growth than other farmers even before any FFS training had taken place. 
This implies that analyses not accounting for this selection bias (such as cross-section 
comparisons, or even the difference-in-differences formulation underlying Table 3) may 
confound the inherent advantage of these select individuals with the program effect. 

The main tests of program impact, comparing changes in the growth rates for exposed 
farmers (Test B) and graduates (Test C) between the pre-program and post-program periods also 
do not provide evidence that yields of these farmers have increased or pesticide use declined 
after exposure to the program. These results do not lend support to the contention that the 
program has been effective. 

Table 5 further checks robustness of the basic findings reported in Table 3 by specifying 
an alternative specification that also accounts for the intensity of FFS activities within a village. 
The intensity of activity (represented by the number of FFS conducted) within a village is 
claimed to be an important factor in the extent to which individual farmers apply FFS principles 
(FAO 1999, p.21). In order to allow for variation in the degree of impact with the intensity of 
FFS activity, in Table 5, the length of post-program exposure for exposed households is 
interacted with the number of FFS conducted in the village.'^ The results support the 
conclusions fi-om Table 3. The tests of program impact (Tests A and B) do not provide evidence 
of improvements in yield growth and reductions in pesticide use for either graduates or exposed 




18 



19 



households. No differences in performance seem to have emerged between graduates and 
exposed farmers after the program (Test C). 

The absence of significant impact that can be attributed to the FFS programs gives rise to 
an obvious question: How can an education effort focusing on matters of direct relevance to 
farmers’ cultivation activities not influence positively their performance? This study cannot 
provide a direct explanation, but some plausible arguments can be advanced: 

(i) Even if graduates gain knowledge in their training that could over time be reflected in 
improved performance, the change is rather small and cannot be detected in the 
econometric study which necessarily imposes a specific structural form that is only 
assumed to reflect reality. The rather impressive gains cited by promoters of the 
program likely exaggerate program impact because of improper attributions, 
confusion of selection biases with true program effects, and extrapolation of 
observations from small non-representative pilot situations and samples to the wider 
population. 

(ii) It is difficult to achieve significant yield gains when there are systemic factors 
causing yield declines such as decline in soil fertility, increased plant diseases, and 
climate trends. Thus, the gains that could be achieved through training may have 
been small to begin with. 

(iii) The knowledge gained in the course of FFS training is complex, as “farmers do not 
master a specific set of contents or ‘messages’, rather, they master a process of 
learning that can be applied continuously” (Dilts). The curriculum of an FFS includes 
agro-ecosystem concepts and decision making principles, that, if conveyed in casual 
and informal conversations, are not likely to be effectively transmitted. Indeed, a 
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recent study of knowledge diffusion related to FFS activities in the Philippines (Rola, 
Jamias, and Quizon) found no significant differences in knowledge scores between 
fanners exposed to FFS graduates and other farmers. If neighbors of graduates do not 
adopt IPM measures, then the pests from their fields can re-infest the fields of 
graduates even if the latter apply IPM principles which reduce initially their 
infestation. This will induce pesticide use even by graduates, and yet the yield may 
decline due to increased pest resistance. 

(iv) During the implementation of the World Bank-financed expansion of the FFS 
program in Indonesia, there were periods when training activities were afflicted by 
untimely transfers of funds to the field training organizers resulting in tr ainin g not 
being fully synchronized with the rice-growing season calendar, irregular supply of 
training materials, and irregular availability of meals for participants. There was a 
relatively large rate of farmer absenteeism in school sessions during part of the 
period. These issues may indicate that the quality of knowledge achieved by 
graduates was adversely affected. These implementation problems are not 
uncommon when pilot projects get scaled-up to the national level. 

(v) As the training program got scaled-up to tens of thousands of villages, a large number 
of trainers had to be inducted into the program. The average quality of trainers and 
their commitment to bottom-up approaches may have been negatively affected in the 
move to mass volume. 

(vi) Pest management, on which the FFS training in Indonesia placed a major emphasis, is 
but a small component of the rice farmers’ cost structure in normal years. Thus the 
cost of pesticides is less than 10% of the total production costs. Farmers are not 
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placing as much importance on what is to them minor production impact as they 
should if the full environmental and health costs of pesticides were to be considered. 
VI. Concluding Observations 

The empirical results of the preceding section do not provide evidence of significant 
improvement in economic performance. And, although the analysis has focused on economic 
benefits, there is no evidence that the expected environmental and health benefits of the program 
are significant, because the empirical results do not indicate a program effect on pesticide use. 
The study shows that it is risky to extrapolate the results of small and early pilots, and the non- 
rigorous analyses that typically accompanied them. The impact of the FFS tr ainin g can be much 
smaller than envisaged, when the program is applied on a large scale, rendering the economic, 
environmental and health benefits much less attractive than what decision makers were 
expecting. 

The discussion of possible reasons for the insignificant program impacts suggests the 
need to exercise more caution in the design of FFS programs and in deciding on whether to scale 
them up to the national level. If the impacts on yields and pesticide use are not large, then the 
viability of the approach depends on maintaining low costs per farmer trained. This is 
particularly true in the case of rice in Asia, where yields are stagnant or falling due to systemic 
factors. The current design is expensive, once all overhead costs are accoimted for.'^ 
Furthermore, the economic viability is undermined by the possibility that diffusion through 
informal communications among farmers is hindered due to the complex nature of the 
information. 

There thus seem to be several implications for improving and reorienting the Farmer 
Field School initiative so as to improve the likelihood of economic viability: 
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(i) There is merit in reviewing the curriculum, and focusing the training on the highest 
priority topics, while simplifying the presentation of the information. There is a 
balance between presenting the operational implications of scientific research and the 
articulation of the science base on which these implications are based.*'* The FFS 
approach may have tilted more than is beneficial into the latter, thus diluting the 
operational effectiveness of the information. 

(ii) The simplification of the program’s content will make it not only more effective with 
respect to improving the performance of graduates, but also increase the likelihood 
and speed of diffusion of new knowledge among other farmers. Diffusion can also be 
enhanced (and made more cost-effective) by employing mass-media and other 
dissemination approaches for key aspects of the knowledge (e.g., safety rules 
regarding the use of chemicals). 

(iii) The narrowing and prioritizing of the curriculum will also shorten the length of the 
training, and will thus cut the cost of the program - a key source of concerns 
regarding fiscal sustainability. With a significant reduction of the per-farmer training 
cost, a much larger number of farmers can be trained directly. When larger groups of 
farmers within a village are trained, there will be better prospects of collective action 
in pursuing coordinated pest management, so that cross-farm infestations do not 
occur. This will make the training more effective in yielding results. 

(iv) Simplification of the ciuriculum, shortening of the training, and increasing the extent 
of simple decision rules in the training will make the program less dependent on 
trainer quality and thus more amenable for scaling up. 
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With changes of the types outlined above, there is less risk and higher prospects of success for 
large scale FFS programs. 
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Econometrics Appendix 



A) Basic difference-in-difference (DD) estimation 

A commonly used approach to measure the impact of FFS on a farm-level outcome is to 
regress the outcome on a variable indicating a farmer’s participation or exposure to the program 
and on other variables that could affect farm outcomes. For example, using the notation from 
section II, log of performance indicator Y for farmer i in village J and time period t is given by 
(!’) ln(7j, ) = a + /3D^y, + + }Xy, + SZy, + A, + 7 . + , 

where X and Z are vectors of household and village characteristics that may change over time, 
and Dg and Du are graduate and exposed household dummy variables. A,, and rjj are unobserved 

determinants of Y that are fixed over time within a household and village respectively, and Ey, is 
a residual that represents all time-varying components of the error. In this framework, // and P 
measure the direct impact of FFS on graduates and the secondary effects on exposed households, 
respectively, under the assumption that the graduate and exposed household dummies are 
orthogonal to the residual. 

However, nonrandom program placement across villages and farmer selection into the 
program are likely to violate the orthogonality assumptions. Nonrandom program placement 
across villages leads to correlation between and rjj , while nonrandom farmer selection into 

the program leads to correlation between Dq and A,. . These correlations imply that and 

P°‘^ are biased estimates of the true causal effect of FFS on graduates and exposed households. 

One solution to the problems of participant selection and program placement bias is to 
estimate probit models that explain which villages and fanners are selected for participation in 
FFS, and then use the Heckman lambda approach (Greene, p. 981) to correct for selection bias in 
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equation (!’). However, this model would be only weakly identified in the Indonesia case, as 
there are no evident instrumental variables that would explain farmer or village participation but 
would have no direct effect on performance. For example, the criteria typically used for selecting 
farmers for training in Indonesia - rice farmer with irrigation, owner-operator, educated, belongs 
to an active farmer group - are also likely to influence farm yields and pesticide use. In the 
absence of valid instruments, the probit selection correction would be identified only by relying 
on a strong distributional assumption of joint normality of the error terms in the selection and 
performance equations in order to identify program impacts; indeed, in the absence of valid 
instmments, it is impossible to identify program impacts with cross-sectional data without 
invoking strong distributional assumptions (Heckman and Robb). 

Weaker identifying assumptions are required when panel data with observations before and 
after the program, for treatment and control groups, are available. With two periods of data, 
household and village level unobservables (A, and rjj) can be eliminated by estimating the 

regression in first difference form. Subtracting 7 in the first period firom the second provides the 
change in Y, conditional on change in program participation; 

(2>) A(ln !;,)=« + + S^Zj, + ^e,., 

= a + 

where AD^ and AD^ are dimuny variables for graduates and exposed households, respectively 
(1 if household graduated firom or was exposed to FFS between the two survey years, 0 
otherwise), while control households are the omitted category.’^ /i {fi) measures the change in Y 
over time for graduates (exposed households) relative to the change in Y for control households. 
This difference-in-difference (DD) estimator provides an unbiased estimate of the effect of FFS 
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on graduates and exposed households under the identifying assumption that change in Y for 
graduates, exposed, and control households would have been the same in the absence of the 
program, even though levels of 7 in any given season may differ (Moffitt).'® Notice that the 
cross-sectional estimator in equation (1’) requires that the levels of Y be the same for graduates, 
exposed, and control households in the absence of the program. The identifying assumption in 
the first difference estimator is a much weaker condition and will more plausibly hold in the 
Indonesia context given the methods of program placement and farmer selection outlined in 
section III. 

B) DD estimation, accounting for variation in length of program exposure 

The specification in equation (2’) can be readily adapted to incorporate the dynamic growth 
process outlined in the conceptual framework. As in the basic DD estimation, the identifying 
assumption is that in the absence of the program, farm performance would have changed at the 
same per period growth rate for controls, exposed and graduate households, and program impact 
on graduates (exposed) can be measured as the change in the per period growth rate for graduates 
(exposed) relative to the per period growth rate for controls. The main difference of the text 
equation (2) with the specification in equation (2’) is that now the graduate and exposed 
household dummies are interacted with the length of program exposure in order to account for 
the fact that the change in outcomes for households that have been exposed for longer periods 
may be greater. 

Incorporating these considerations (and suppressing subscripts to simplify the exposition) 
leads to: 

(3’) A(ln Y) = a{r' - T , )+ J3D^ (r, - T* )+ (t, - T* )+ jAY + + Ae 
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where To and Tj refer to the two survey years, and T* is the year in which graduates mderwent 
training within the village, such that Tq<T‘ a measures the pre-program exposure 

growth rate in Y (assumed to be the same for graduates, exposed and control households in 3 ’). // 
and P estimate the growth rates in Y for graduates and exposed household after they were 
exposed to FFS. Since the growth rate of Y for control households is measured by oc, program 
impact on graduates and exposed households is simply ijl-cY) and ifi-ci), respectively. It can be 
shown that if the program were introduced at an identical time in all villages (in which case T* is 
identical across observations), then the specification in (3’) reduces to the standard difference-in- 
difference formulation, with the parameters scaled up by the fixed factor {TrT*). 
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Table 1. Descriptive statistics, by farmer category 





Controls 


Farmer Category 
Exposed 


Graduates 


I. Household Composition (1990/91) 
Household size 


4.48 


4.74 


4.69 




(1.51) 


(1.92) 


(1.52) 


% adult males 


0.29 


0.25 


0.32 




(0.11) 


(0.17) 


(0.18) 


% adult females 


0.25 


0.29 


0.29 




(0.15) 


(0.15) 


(0.15) 


IL Household Assets (1990/91) 
Land area owned (ha) 


0.61 


0.71 


1.24 




(0.52) 


(1.12) 


(2.73) 


% sawah area rainfed 


0.58 


0.27 


0.13 




(0.50) 


(0.43) 


(0.33) 


No. of 4 wheel and hand tractors 


0.06 


0.05 


0.06 




(0.42) 


(0.25) 


(0.28) 


No. of sprayers 


0.46 


0.46 


0.63 




(0.61) 


(0.59) 


(0.83) 


No. of years of education of household head 


5.71 


5.15 


5.77 




(2.61) 


(2.71) 


(2.70) 


IIL Participation in Community Activities (1998/99) 
% who work as a village officer 


0.13 


0.10 


0.29 




(0.34) 


(0.34) 


(0.45) 


% in farmer groups 


0.81 


0.55 


0.82 




(0.40) 


(0.50) 


(0.38) 


% in cooperatives 


0.19 


0.16 


0.24 




(0.40) 


(0.37) 


(0.43) 


IV. Number of households 


52 


156 


112 



Notes. - Standard deviations reported in parentheses. 
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Table 2. Sample averages of outcome indicators, by farmer category 





1990/91 


1998/99 


Change between 1990/91 and 1998/99 








absolute diff. 


% change 


Yield (kg/ha) 
Control fanners 


6620.2 


5594.5 


-1025.8 


-15% 




(1153.6) 


(1451.5) 


(1478.0) 




Exposed farmers 


5757.3 


5326.3 


-431.1 


-7% 




(1369.7) 


(1567.1) 


(1783.1) 




FFS graduates 


6116.3 


5472.0 


-644.3 


-11% 




(1194.5) 


(1376.0) 


(1579.0) 




All farmers 


6023.2 


5420.9 


-602.7 


-10% 




(1310.1) 


(1482.9) 


(1675.1) 




Java Average (from 
National statistics) 


5104.0 


4849.0 


-255.0 


-5% 


Pesticide ('000s of 1998 Rp/ha) 








Control farmers 


65.6 


176.5 


110.9 


169% 




(66.7) 


(265.7) 


(275.1) 




Exposed farmers 


92.8 


202.0 


109.1 


118% 




(91.6) 


(236.7) 


(233.8) 




FFS graduates 


111.3 


201.9 


90.6 


81% 




(108.6) 


(184.3) 


(177.3) 




All farmers 


94.9 


197.8 


102.9 


109% 




(95.6) 


(224.5) 


(222.9) 





Notes. - Standard deviations reported in parentheses. CPI used to inflate 1990/91 pesticide expenditures 
to 1998/99 Rupiah. 
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Table 3, Impact of FFS on Rice Yields and Pesticide Use 



Parameters 


Yield 


Pesticides 


Pre-orogram growth rates 






All Farmers (a) 




-0.028 


0.251 






(2.85)^ 


(5.29)^ 


Post-nrogram growth rates 






Exposed farmers (p) 




-0.033 


0.365 






(3.28)^ 


(4.84)^ 


Graduates, after own 


training (p) 


-0.036 


0.362 






(3.22)^ 


(4.32)* 


R* 




0.34 


0.37 


Hypotheses Tests (n 


-values): 






Yield 


Pesticides 


ProbfT > t) 


Probrr < t) 


A: p > a 


3 < a 


0.854 


0.994 


B: p, > a 


p, <a 


0.902 


0.984 


C: u >3 


<P 


0.810 


0.445 



Notes: Dependent variables are the differences in logs of household-level yields and pesticide use on the main rice 
plot between the 1998/99 and 1990/91 rice growing seasons. Both regressions control for household and village 
characteristics and district dummies. These sets of variables are jointly significant at 5%. Absolute value of robust 
t-stats, corrected for clustering within villages, reported in parentheses. ♦ significant at 5%. Sample size in both 
regressions is 320. 
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Table 4. Robustness check: allowing for differences in pre-program growth rates 



Parameters 


Yield 


Pesticides 


Pre-Dro 2 ram growth rates 






Exposed and Control 


Farmers (tti) 


-0.006 


0.141 






(0.58) 


(2.59)^ 


Graduates (a 2 ) 




0.005 


0.084 






(0.52) 


(1.22) 


Post-program growth rates 






Exposed farmers (p) 




-0.012 


0.260 






(1.16) 


(2.88)^ 


Graduates, after own 


training (p.) 


-0.017 


0.266 






(1.45) 


(2.79)^ 






0.35 


0.37 


Hypotheses Tests (n 


-values): 






Yield 


Pesticides 


ProbfT > t) 


ProbfT < t) 


A: tt2 > tti 


A: tt2 < tti 


0.000^ 


0.006^ 


B: p>ai 


B: p < tti 


0.895 


0.996 


C: u > ci 2 


C: n < tt2 


0.999 


0.998 



Notes: Dependent variables are the differences in logs of household-level yields and pesticide use on the main rice 
plot between the 1998/99 and 1990/91 rice growing seasons. Both regressions control for household and village 
characteristics and district dummies. These sets of variables are jointly significant at 5%. Absolute value of robust 
t-stats, corrected for clustering within villages, reported in parentheses. ♦ significant at 5%. Sample size in both 
regressions is 320. 
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Table 5. Robustness check: allowing for differences in number of FFS 



Parameters 


Yield 


Pesticides 


Pre-oroeram growth rates 
All Farmers (a) 


0.006 


-0.120 




(1.06) 


(2.07)* 


Post-uroeram growth rates 
Exposed fanners A’No. of FFS (P) 


-0.001 


0.007 




(0.78) 


(0.71) 


Graduates, after own training (p) 


-0.005 


0.011 




(1.08) 


(0.42) 




0.34 


0.36 


HvDOtheses Tests tD-values^: 
Yield Pesticides 


ProbfT > t) 


ProbfT < t) 


A: P > a P < a 


0.908 


0.983 


B: |i >a |i <a 


0.945 


0.973 


C: ^ > p ^ < p 


0.853 


0.568 



Notes: Dependent variables are the differences in logs of household-level yields and pesticide use on the main rice 
plot between the 1998/99 and 1990/91 rice growing seasons. Both regressions control for household and village 
characteristics and district dummies. These sets of variables are jointly significant at 5%. Absolute value of robust 
t-stats, corrected for clustering within villages, reported in parentheses. ♦ significant at 5%. Sample size in both 
regressions is 320. 
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Endnotes 



* It is important to note that in this study, the analysis is limited to measurable economic outcomes - i.e., yields and 
pesticide use which are two performance indicators frequently highlighted by promoters of the program. The 
analysis does not deal with other effects that field school training is claimed to bring about such as improved health, 
enhanced political activism, a sense of pride and empowerment, and build-up of social capital (Dilts; Roling and van 
de Fliert). 

^ Modeling performance as a dynamic process is compatible with the way sociologists and economists perceive the 
process of innovation uptake. For example, Rogers describes the process of innovation adoption as “the mental 
process an individual passes from first hearing about an innovation to final adoption.” Similarly, the leaming-by- 
doing literature suggests an evolution of performance over time (Feder, Just, and Zilberman, p. 259). The dynamic 
performance model is also compatible with Bayesian information-updating processes, describing the way 
perceptions regarding technology parameters change over time with improved information (Feder and Umali, p. 

216). 

^ While the conceptual framework is applicable more broadly to a range of farm-level outcomes, in the empirical 
work we focus specifically on rice yields and pesticide use on the main rice plot, as the improvements in rice yields 
and reduction in pesticide use have been claimed to be important economic achievements of the Indonesian FFS 
program (Kenmore 1997; FAO 2000). 

^ Note that even if the “without program” trend in yields is negative (due to external factors affecting the whole farm 
sector such as systemic climate changes or declining soil fertility), the program’s impact is expected to be positive in 
that it enables participants (and their neighbors) to slow the rate of decline compared to control farmers. Similarly, 
even if the overall “without program” trend is one of increased pesticide use (due to factors such as increased pest 
resistance), the program is expected to slow the growth of pesticide use by graduates and exposed farmers compared 
to control farmers. 

* In the empirical analysis, we do not differentiate between graduates trained by officials versus farmer-trainers 
because previous studies have shown that the quality of training provided by the two sources was similar (van de 
Fliert, Pontius, and Roling). 

^ For a more detailed discussion of the econometric issues in our context, please see the Appendix. In brief, the 
econometric problem caused by non-random program placement and participant selection is that if selection is based 
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on unobservable household and regional traits that also affect the outcomes of interest, then the participation 
variables in a cross-sectional regression of outcomes on participation are correlated with the error term. Therefore 
the estimates of parameters associated with these variables are biased. The DD or first-difference estimator 
eliminates this source of endogeneity as long as the unobservables are time-invariant. 

^ Household controls include plot size and irrigation, education of household head, and household size and 
composition. Village-level variables control for availability of different types of infrastructure and services, 
including cooperatives, schools, observers, production input kiosks, irrigation, road quality, and distance from 
district center. District-level differences are controlled for by including district dummies. 

® While it may be argued that the passage of time could cause “erosion” of knowledge, this seems to be less likely in 
the case of knowledge gained through FFS, as the program is said to have engendered communal activities and 
continuous focus on FFS themes through vibrant alumni associations (FAO 1999, pp. 19-21). 

^ A secular decline in Asian irrigated rice yields over time has been observed by researchers in recent years (e.g., 
Cassman, Oik, and Dobermann), and is attributed to declines in soil fertility, increased resistance to pesticides, and 
increased vulnerability to diseases such as sheath blight and blast. In addition, global warming has been associated 
with a decline of Java rice yields (Amien et. al.). Finally, the 1998/99 rice season in Indonesia was mildly affected 
by La Nina. While these factors negatively impact on all farmers, it is expected that farmers trained in FFS (and 
those who learn from them) will have experienced smaller declines due to their better knowledge of pest and crop 
management. 

The only exception is the difference in change in yields, between exposed and control group households. Yields 
for the control group declined by significantly more than the decline observed for the exposed households. 

" Dependent variables in the regressions are the differences in log(yield) and log(pesticide use per hectare) on the 
main rice plot between the main 1998/99 and 1990/91 rice growing seasons. 

Post-program exposure for graduates is not interacted with number of FFS since it is most likely that the main 
effect on their performance is related to the intensive training they have received and the diffusion through informal 
conversation is of second order magnitude. 

A conservative estimate of the cost per farmer trained is approximately $49 per farmer (Quizon, Feder, and 
Murgai). 
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Pingali, Hossain, and Gerpacio (pp. 267-271) discuss the trade-off between costly and intensive IPM training and 
cheaper dissemination of simple science-based messages. 

Note that the change in participation variables are the same as the dummy variables defined in equation (T) since 
none of the households in the sample had graduated or been exposed to FFS in the first survey year. That is, 

= Dq and AD^ = . 

This first difference estimator is often termed a dijference~in~dijferences estimator because it computes the 
program impact by comparing the change in outcomes for participants to the change in outcomes for non- 
participants. 

Interacting the participation dummies with the length of exposure variables implies that the three variables 
attached to parameters a; and // are specified as follows for the three groups: (a) for all control households, the 
variables take the values (Ti-To), 0 and 0; (b) for exposed households, the variables are (T^-To), (T^i-T^), and 0, 
where 7^ varies across exposed farmers depending on the season when the first FFS was held in the village; (c) for 
graduates, the variables are (7^-7b), 0, and (Tp 7^) where 7^ varies across graduates depending on when the 
graduate attended a FFS, respectively. 
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Figure 1. The Evolution of Performance over Time 
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